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(1) Real Party in Interest 

Silicon Graphics, Incorporated was the original assignee, and subsequently 
assigned all right and title to the subject invention to the real party in interest, the 
Microsoft Corporation. 

(2) Related Appeals and Interferences 

Appellant is not aware of any other appeals or interferences which will 
directly affect, be directly affected by, or otherwise have a bearing on the Board's 
decision to this pending appeal. 

(3) Status of Claims 

Claims 1-20 were originally submitted. Claim 2 is canceled. Claims 1,3, 
16, and 20 were amended. Claims 1, 3-20 stand rejected and are pending in this 
Application. All pending claims are set forth in the Appendix of Appealed Claims 
on page 22. 

Claims 1, 4-5, 9, 12-14, 16, 18 stand rejected under 35 U.S.C. §103(a) as 
being unpatentable over U.S. Patent No. 5,822,537 to Katseff et al (hereinafter, 
"Katseff ') in view of Shibata, Y., "Media Synchronization Protocols for Packet 
Audio- Video System on Multimedia Information Networks", IEEE, January 3-6, 
1995, pp. 594-601 (hereinafter, "Shibata"). 

Claim 3 stands rejected under 35 U.S.C. §103(a) as being unpatentable over 
Katseff, and Shibata and in further view of. U.S. Patent No. 5,642,171 to 
Baumgartner et al (hereinafter, "Baumgartner"). 

Claims 6, 7, 10, and 17 stand rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Katseff, and Shibata and in further view of U.S. Patent No. 
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5,950, 202 to Durward et al (hereinafter, "Durward"). 

Claim 19 stands rejected under 35 U.S.C. §103(a) as being unpatentable 
over Katseff, in further view of Durward. 

Claims 8, 11, and 15 stand rejected under 35 U.S.C. §103(a) as being 
unpatentable over Katseff, and Shibata and in further view of U.S. Patent No. 
5,812,791 to Wasserman et al (hereinafter, "Wasserman"). 

Claim 20 stands rejected under 35 U.S.C. § 103(a) as being unpatentable 
over Katseff, Shibata, Durward in further view of Baumgartner. 

(4) Status of Amendments 

All amendments have been entered, and are reflected in the Appendix of 
Appealed Claims. 

(5) Summary of Invention 

The claimed invention is directed to synchronizing asynchronous time- 
based and motion data by retrieving from a server frames of data that make up a 
time-based data stream and a motion data stream, variably buffering one of the 
two streams, and producing two streams with synchronized frames. The two 
streams with synchronized frames are played at a client, producing synchronized 
motion and time-based data. See page 2, lines 1-12 of the specification. 

Synchronization of the data frames (data streams) takes place at a 
synchronizer prior to transmitting the synchronized data streams to a network, 
eliminating the need for the client to synchronize the data streams. See page 9, 
lines 3-6 of the specification. 
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Fig. 2 of the present application is representative of the system and method 
and is reproduced below: 
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Figure 2 

Fig. 2 shows a server 100, and a client 102 connected by a network such as 
the Internet 103. Time based data includes voice data from a microphone 114 
and/or from a prerecorded audio storage device 110. Motion data includes data 
from a body suit 116 or similar sensory device. Motion data may be integrated 
with other data through a keyboard 118, slider 120, and/or joystick 122. See page 
1, lines 18-31 of the specification. Furthermore, the received motion data may be 
divided into one or more groups that include one or more channels. A channel 
group is defined as a group of related sensor inputs. See page 7, lines 16-20 of the 
specification. 

Server 100 includes a data synchronizer 200, an object list module 201, a 
packetizer 202, a playback and record module 204, a multicaster 206, a time-based 
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data interface 208, and a motion data interface 210. Server 100 may multicast 
synchronized motion and time-based data to one or more clients, such as client 
102. See page 6, lines 22-27 of specification. The motion data interface 210 
includes a user interface, drivers and a mapper. The mapper receives input user 
preferences through the user interface for mapping the various different input 
motion data streams to channels in an output motion data stream which is provided 
as an input to data synchronizer 200. See page 8, lines 18-20, 26-31 of 
specification. 

Client 102 includes a packet receiver 220, a packet sequencer 222, a 
deserialization unit 224, a reader 225, an audio interface 226, and a motion data 
interface 228. See page 6, lines 26-29 of specification. 

Data synchronizer 200 receives as an input both time-based data input 
streams and output motion data streams, and provides synchronized frames of 
motion and time-based (audio) data to object list module 201. Voice data, such as 
voice data from microphone 114 provided to time based data interface 208, is 
synchronized with motion data by data synchronizer 200 in order that the audio 
and display (motion) data displayed at the client 102 appears synchronized. See 
page 9, lines 4-16 of specification. Furthermore, data synchronizer 200 receives 
as an input from sensory devices such as bodysuit 116 a time stamp and data on 
one or more channels. Motion sensors on devices such as bodysuit 116 may 
include one hundred and forty channels of data and two channels for timing 
information where the timing information (time stamp information for 
synchronizing the data streams) is used for synchronizing data streams. In other 
words, the time stamps (timing information) may be used to calculate delay 
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differences between time-based (audio) data and motion data. See page 10, lines 
21-26 of specification. 

Data synchronizer 200 further includes buffers and delay elements used to 
synchronize motion and time-based (audio) data streams. See page 9, lines 28-32 
of specification. 

During an initialization state, the motion data interface 210 retrieves one or 
more motion data streams and assigns data channels to the input streams resulting 
in an input channel mapping for data in an output motion data stream. See page 
11, lines 1-5 of specification. 

Delay (i.e., synchronization) is calculated based on time stamp information 
provided with the data for each stream or channel group. See page 12, lines 25-27 
of specification. 

(6) Issues 

Whether claims 1, 4-5, 9, 12-14, 16, 18 are properly rejected under 35 
U.S.C. § 103(a) as being unpatentable over Katseff in view of Shibata. 

Whether claim 3 is properly rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Katseff, and Shibata, and in further view of Baumgartner. 

Whether claims 6, 7, 10, and 17 are properly rejected under 35 U.S.C. 
§ 103(a) as being unpatentable over Katseff, and Shibata, and in further view of 
Durward. 

Whether claim 19 is properly rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Katseff in view of Durward. 

Whether claims 8, 1 1, and 15 are properly rejected under 35 U.S.C. §103 (a) 
as being unpatentable over Katseff, and Shibata, and in further view of 
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Wasserman. 

Whether claim 20 is properly rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Katseff, Shibata, and Durward, and in further view of 
Baumgartner. 

(7) Grouping of Claims 

Appellant respectfully submits that the rejected claims Claims 1, 3-20 do 
not stand or fall together. The set of pending claims are separated into two 
groupings of claims, identified below as groups A and B. As is explained below, 
the groupings of the claims are separately patentable. However, it should be 
understood that the claim groupings below are presented for the purposes of 
isolating and reducing issues for this Appeal; thus, the claim groupings below 
should not be considered as the only groupings nor should individual claims be 
considered as consequentially not separately patentable. 

A, Claims 1, 3-10, 12, 13, 16, 17, 19, and 20 

B, Claims 11, 14, 15, and 18 

(8) Argument 

All of the submitted claims were rejected in the Office Action of 02/04/03 
based upon Katseff; and in further view of one or more of the references Shibata, 
Durward, Wasserman, and Baumgartner. 

Katseff shows a multimedia system to record, store and distribute 
multimedia presentations together with materials that may be referenced during 
the presentation. See Abstract of Katseff, Katseff shows a continuous media and 
storage retrieval system used for digitizing and compressing audio, video, and, 
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other continuous data. See Katseff at col 5 lines 18-22. Katseff also shows a 
digitizer/compressor that generates a data stream that preferably is in JPEG file 
format. JPEG file format is preferred because of the inherent synchronization of 
audio and video streams. See Katseff at col 6, lines 60-67 through col 7, lines 1- 
7. Katseff further shows an information retrieval system that allows a user to 
access multimedia information that has been stored in a plurality of databases (i.e., 
file servers). See Katseff at col 8, lines 18-22. 
Figure 1 of Katseff is redrawn below: 
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Katseff shows distinct workstations or client computers (clients) 15, 25 5 and 
30. Clients 15, 25,. and 30 are connected to a network 20. Network 20 further 
connects to file servers 35 and 40. 
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Information retrieval system 50, still image storage and retrieval system 60, 
and continuous media storage and retrieval system 70 are included in a networked 
multimedia system 10. 

Continuous media storage and retrieval system 70 includes a continuous 
media and storage capture system used to obtain an electronic representation of 
each continuous media object and to store the electronic representation on a file 
server such as file server 40. See Katseff, col 5, lines 32-37. 

Representative client 15 includes an audio buffer 110 and a video buffer 
115. As described in Katseff, audio buffer 110 and video buffer 115 are used to 
address congestion on network 20. See Katseff, col 14 line 56 to col 16 line 57. 

Katseff discloses that it is preferred that the workstation (or client) will 
have a mechanism to synchronize the presentation of the various outputs (audio 
and video). See Katseff at col 9, lines 3-5. Katseff does not disclose that audio 
and video frames be synchronized prior to receipt at the workstation. 

In Katseff, a workstation (client) retrieves several frames of audio and 
video from a file server (e.g., file servers 35, 40) for storage in an audio buffer and 
a video buffer at workstation. See Katseff at col 8, lines 60-64. No buffering and 
synchronization is made as to the audio and video frames at the server. As 
discussed above, a client has audio and video buffers. 

Shibata shows a rate controller at a video server and a rate controller at a 
client station. The rate controller at the video server updates a current frame rate 
based on a control message. The rate controller at the client station also updates 
the frame rate to the new value (frame rate) that is sent from the video server. 
Repeating this frame rate update allows the client station to adjust and maintain a 
constant rate. Shibata particularly points out that this process only applies to video 
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rate, since the audio segments do not require rate control, but are instead sent with 
higher priority than the video frame to minimize any audio delay. See page 596 of 
Shibata. 

Baumgartner shows a method of synchronizing audio and video data. 
Baumgartner relies on a common starting point for the audio and video data. See 
col. 13, lines 48-50 of Baumgartner. Since an audio data stream and video data 
stream have a common starting point, a synchronization error quantity may be 
calculated to determine the number of frames out of sync that the audio data 
stream or the video data stream is from one another. See col 13, lines 63-66 of 
Baumgartner. In particular, the method shown in Baumgartner shows a 
determination of frame advance of audio data over video data. See col 14, lines 1- 
9 of Baumgartner. 

Durward shows an instrumented garment used to provide input to define a 
virtual being within a virtual space. The instrumented garment is used along with 
other input devices such as a keyboard, microphone, and a head mounted display 
and sensors to define the virtual being and the virtual space. See col 3, lines 5-31 
of Durward. Durward further shows a user logging onto a central control unit in 
order to transmit data to the central control unit which includes mapping 
information from the user to create a virtual space. The central control unit relies 
on input from various users. The central control unit sends back a particular 
virtual space to a particular user. Therefore users may or may not receive the 
same motion and sound data. See col 8, lines 22-25, 61-64 of Durward. 

Wasserman shows a system used to decompress varied image sizes, 
multiple threads, and moving video, and still images for use in a multimedia 
application. The moving video and still images are separate from one another. 
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The still images may be used as a background for the moving video. See col 6, 
lines 8-16 of Wasserman. 

The following arguments are organized into two sub-arguments, one for 
each of the claim groupings. 

(1) The Cited Combination of Katseff and Other References Does 
Not Teach or Suggest Synchronizing Time-based and Motion 
Data at a Server, 

All claims in grouping A (claims 1, 3-10, 12, 13, 16, 17, 19, and 20) stand 
rejected under 35 U.S.C. § 103(a) as being unpatentable over Katseff in view of 
one or more of the following : Shibata, Durward, Wasserman, and Baumgartner. 

Claim 1 is representative of claim grouping^. 

Claim 1 recites: 

A method of synchronizing asynchronous time-based and motion 
data in a system in which the time-based data and the motion 
data are transmitted by a server over a network to a client, the 
method comprising: 

retrieving a time-based data stream and a motion data stream at the 
server, each stream comprising frames of data; 

variably buffering one of the time-based data stream and the motion 
data stream at the server to produce two streams having 
synchronized frames; and 

using the synchronized frames at the client for playback of 
synchronized motion and time-based data to a user. 

Katseff does not teach "retrieving a time-based data stream and a motion 
data stream at the server"; "variably buffering one of the time-based data stream 
and the motion data stream at the server to produce two streams having 
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# 



synchronized frames"; and "using the synchronized frames at the client for 
playback of synchronized motion and time-based data to a user". 

Although Katseff shows audio and video buffers that receive audio and 
video data, such buffers are located at a client and not a server. The client in 
Katseff performs synchronizing of audio and video data, not the server. The 
servers in Katseff merely store audio and/or video data that may be accessed by a 
client. 

The element of "retrieving a time-based data stream and a motion data 
stream at the server" is specifically recited because the server performs 
synchronization as recited in the element "variably buffering one of the time-based 
data stream and the motion data stream at the server to produce two streams 
having synchronized frames". This alleviates the need for clients to perform 
synchronization of audio (time based) and video (motion based) data streams. 

Katseff does disclose that a server accesses a JPEG movie file that 
interleaves a frame of audio and video. The Examiner has interpreted this as the 
server "accessing of separate audio (time-based) and video (motion-based) data." 
The Examiner argues that a JPEG movie file interleaves a frame of audio and 
video (which can be interpreted as a movie separated into separate audio and video 
component frames) (Katseff column 6, lines 64-67). Applicant does not disagree 
with the Examiner's definition of a JPEG movie file; however, a JPEG movie file 
is a single data stream. That single data stream has interleaved frames of audio 
and video. Katseff particularly mentions a JPEG movie file format, because of its 
inherent synchronization of audio and video streams. If a server receives such a 
JPEG movie file format, there would not be a need to synchronize the audio and 
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video data contained in such a JPEG movie file, since the JPEG movie file format 
seems to address the synchronization of the audio and video streams. 

In contrast, claim 1 recites retrieving both a time-based data stream and a 
motion data stream at the server. These are two distinct streams which are 
retrieved at the server. Katseff does not disclose or teach that two distinct streams, 
time-based stream and motion data stream, are retrieved from a server. 

The Examiner notes that Katseff discloses "a method for relieving network 
congestion by monitoring buffers' threshold and compensating by reducing video 
transmittal rate, then reducing audio playback rate" (Katseff - title). As it 
suggests, Katseff is directed to "monitoring buffers' threshold" to relieve network 
congestion. Katseff does not disclose or teach buffering to produce two streams 
with synchronized frames. While Katseff discloses use of audio and video buffers, 
these buffers are located at the client and not at the server (Katseff, column 8, lines 
60-67). Since Katseff uses an inherently synchronized file format, it does not need 
buffers at the server to "variably buffer the two streams at the server". 

Examiner points out that Shibata teaches a rate control message that is sent 
to a server. The rate controller described on page 596 of Shibata does not suggest 
that data streams be variably buffered. Shibata' s rate controller monitors a current 
video frame rate with a set video frame rate value and adjusts the video frame rate 
depending on the difference between the set video frame rate value and the current 
video frame rate value. Further Shibata does not suggest that audio or time-based 
rate values can be controlled. Shibata states that "since the audio segment rate is, 
relatively, much smaller than that of the video, audio segments do not require rate 
control ... [Tjherefore, only the video frame rate is controlled." (Shibata at page 
596). 
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Shibata provides no assistance in light of Katseff as to the recited 
methodology of claim 1. Accordingly, a combination of Katseff and Shibata fails 
to teach or suggest the claimed methods. 

In this case, the prior art discloses no advantages or utility for the proposed 
combination. Accordingly, the combination proposed by the Examiner would not 
have been obvious, and the rejection of claim 1 is unfounded. Allowance of claim 
1 is respectfully requested. 

Claims 4, 5, 12, and 13 of Group A all depend from claim 1, and are 
allowable because of their dependence from an allowable base claim. 

Claim 16 of Group A recites "[a]n apparatus resident on a server for 
synchronizing asynchronous time-based and motion data"; "retrieving a time- 
based data stream and a motion data stream at the server"; and "buffering one of 
the time-based data stream and the motion stream to produce two streams having 
synchronized frames"; and benefit from the same arguments of claim 1 . 

Claim 3 of Group A stands rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Katseff and Shibata, and in further view of Baumgartner. Claim 
3 depends from claim 1 and is allowable because of its dependence from an 
allowable base claim. Baumgartner relies on a common starting point for audio 
and video data in calculating a difference and counting frames' of data. The 
present invention relies on a time stamp and determining difference in calculating 
any delays between the motion and time-based data streams. 

In this case, the prior art discloses no advantages or utility for the proposed 
combination. Accordingly, the combination proposed by the Examiner would not 
have been obvious, and the rejection of claim 3 is unfounded. 

Claims 6, 7, 10, and 17 of Group A stand rejected under 35 U.S.C. § 103(a) 
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as being unpatentable over Katseff and Shibata, and in further view of Durward. 
Claims 6, 7, and 10 depend from claim 1 and are allowable because of their 
dependence from an allowable base claim. Claim 17 depends from claim 16 and 
is allowable because of its dependence from an allowable base claim. 

Durward shows an instrumented garment to provide motion data input; 
however, Durward does not show that such motion data may be synchronized with 
separate time based data such as audio. The motion data input of Durward merely 
is used to define a virtual being within a virtual space. Furthermore, Durward 
relies on users providing input (transmit data) to a central control unit in order 
create a virtual space. Durward does not show the element of claim 17 of "a 
multicaster for multicasting the synchronized motion and time-based data". The 
same synchronized motion and time-based data is the sent or multicast to multiple 
clients (users). Durward teaches that depending on the user's inputs, the same 
motion and sound data may not be sent (broadcasted) to the same users, since the 
intent of Durward is to create a customized virtual space for each user. 

In this case, the prior art discloses no advantages or utility for the proposed 
combination. Accordingly, the combination proposed by the Examiner would not 
have been obvious, and the rejection of claims 6, 7, 10, and 17 is unfounded. 

Claim 19 of Group A stands rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Katseff, in further view of Durward. Claim 19 of Group A 
recites "playing back time-based and motion based data that has been 
synchronized" and benefits from the same arguments of claim 1 in regards to 
Katseff. Claim 19 further recites "mapping the motion based data to control the 
movement of a virtual figure displayed in a scene displayed at a client" and 
"playing back in synchronization with the movement of the virtual figure the time- 
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based data." Durward shows a virtual figure created by an instrumented garment 
and other devices; however, Durward does not show that such a virtual figure may 
be controlled by motion based data or its movement synchronized with time-based 
data. 

In this case, the prior art discloses no advantages or utility for the proposed 
combination. Accordingly, the combination proposed by the Examiner would not 
have been obvious, and the rejection of claim 19 is unfounded. 

Claim 8 of Group A stands rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Katseff, and Shibata and in further view of Wasserman. Claim 
8 depends from claim 1 and is allowable because of its dependence from an 
allowable base claim. Claim 8 further recites "motion data that includes 
background data for use in producing a scene at the server". Wasserman particular 
shows a distinct moving video (motion data) and distinct still images. The still 
images may be used as background fro the moving video. Wasserman does not 
teach that the moving video includes the still images (background data). 

In this case, the prior art discloses no advantages or utility for the proposed 
combination. Accordingly, the combination proposed by the Examiner would not 
have been obvious, and the rejection of claim 8 is unfounded. 

Claim 20 of Group A stands rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Katseff, Shibata, Durward, and in further view of Baumgartner. 

Claim 20 of Group A recites "retrieving an audio stream including voice 
data and a motion data stream"; and "variably buffering a faster of the streams to 
synchronize the audio stream and the motion data stream resulting in two output 
streams having synchronized data frames" and benefits from the same arguments 
of claim 1. Claim 20 further recites "calculating a difference between the delay 
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for the audio stream and the motion data stream to determine which of the two 
streams is faster". The Examiner cites Baumgartner as teaching this element; 
however, as discussed above Baumgartner is limited by its reliance on a common 
starting point to determine the difference. Claim 20 further recites "multicasting 
the synchronized data frames to one or more clients over a network". The 
Examiner relies on Duward as teaching this element; however, as discussed above 
Durward provides particular virtual space data to users depending on the input 
provided by the users. 

(2) The Cited Combination of Katseff and Other References Does Not 
Teach or Suggest Channels for Data Frames. 

All claims in grouping B (claims 11, 14, 15, and 18) stand rejected under 35 
U.S.C. § 103(a) as being unpatentable over Katseff in view of one or more of the 
following : Shibata, Durward, Wasserman, and Baumgartner. Claim 1 1 depends 
from claim 1, and claim 18 depends from claim 16. Accordingly claims 11 and 18 
in grouping B are allowable for the reasons discussed above with regard to claim 
grouping yl. 

Claim 14 is representative of claim grouping 5. Claim 14 is directed 
generally to packaging synchronized frames of data where each frame of data 
includes one or more channels. 
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Claim 14 recites: 

A method of packaging synchronized frames of data where each 
frame includes one or more channels of data in a system in 
which synchronized frames are transmitted by a server over a 
network to a client, the method comprising: 

storing a last data value for each channel in each frame transmitted 
over the network; 

retrieving new synchronized frames for transmission over the 
network; and 

packaging and transmitting over the network only data for channels 
having changed data values. 

The cited references of Katseff, Shibata, Durward, Wasserman, and 
Baumgartner do not show "storing a last data value for each channel in each frame 
transmitted over the network". 

Katseff describes a server and client retrieving data streams such as audio, 
video, and JPEG data. Such data streams include frames; however, Katseff does 
not show that such frames may include one or more channels. Shibata is cited for 
its rate controller that adjusts video frame rates, but Shibata fails to disclose that 
such frames may include one or more channels. The cited references of 
Baumgartner, Durward, and Wasserman describe retrieving data (frames); 
however, none of these references disclose that the data (frames) may include one 
or more channels. Since the references do not show frames of one or more 
channels, the references fail to show the recited element of "storing a last data 
value for each channel in each frame transmitted over the network". 

Furthermore, the cited references do not show "packaging and transmitting 
over the network only data for channels having changed data values". As 
discussed above, the cited references of Katseff, Shibata, Baumgartner, Durward, 
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and Wasserman describe frames and data; however, none of the references show 
that such frames or data to include one or more channels. Therefore, the 
references do not show the recited element of "packaging and transmitting over 
the network only data for channels having changed data values". 

Claim 15 depends from claim 14 and is allowable because of its 
dependence from an allowable base claim. 

Claim 1 1 of group B recites "synchronized data frames include one or more 
data channels" and benefits from the same arguments of claim 14. 

Claim 18 of group B recites "each channel in each of the synchronized 
frames" and benefits from the same arguments of claim 14. 
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Conclusion 

Appellant respectfully requests that the §103 rejection be withdrawn and 
that pending claims 1, 3-20 be allowed. 

Respectfully Submitted, 

Dated: jUsfa By: \I(L^ 

' EmmanUe? A. Rivera, Reg. No. 45,760 

(509) 324-9256 
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(9) Appendix of Appealed Claims 

1. A method of synchronizing asynchronous time-based and motion 
data in a system in which the time-based data and the motion data are transmitted 
by a server over a network to a client, the method comprising: 

retrieving a time-based data stream and a motion data stream at the server, 
each stream comprising frames of data; 

variably buffering one of the time-based data stream and the motion data 
stream at the server to produce two streams having synchronized frames; and 

using the synchronized frames at the client for playback of synchronized 
motion and time-based data to a user. 

3. The method of claim 1 further including calculating a difference 
between delays for the motion data stream and the time-based data stream through 
the server to determine an amount of variable buffering for a faster of the two 
streams. 

4. The method of claim 1 further including transferring only those data 
values for a frame that have changed since a last frame was transmitted. 

5. The method of claim 1 wherein the network is the Internet. 

6. The method of claim 1 wherein the motion data is mapped to control 
the movement of a virtual figure displayed in a scene at the client. 
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7. The method of claim 1 wherein the motion data is generated by a 
body suit. 

8. The method of claim 1 wherein the motion data includes background 
data for use in producing a scene at the server. 

9. The method of claim 1 wherein data transfer from the server to the 
client is concurrent with the receipt of the time-based data stream and motion data 
stream at the server. 

10. The method of claim 1 wherein the time-based data is voice data. 

11. The method of claim 1 wherein the synchronized data frames 
include one or more data channels, the server transmitting on the network at a 
predetermined interval between synchronized data frames a descriptor packet 
which describes each channel contained in the synchronized data frames such that 
a client may join in progress a multicast of synchronized data frames. 

12. The method of claim 1 wherein the time-based data is a pre-recorded 
audio track and the method further includes synchronizing playback of the pre- 
recorded audio track at the server and buffering of the pre-recorded audio track to 
allow for coupling with motion data generated in time with the playback of the 
pre-recorded audio track. 
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13. The method of claim 1 further including sequencing synchronized 
frames output from the server to the client to provide for ordered playback of the 
synchronized frames to a user at the client. 

14. A method of packaging synchronized frames of data where each 
frame includes one or more channels of data in a system in which synchronized 
frames are transmitted by a server over a network to a client, the method 
comprising: 

storing a last data value for each channel in each frame transmitted over the 
network; 

retrieving new synchronized frames for transmission over the network; and 
packaging and transmitting over the network only data for channels having 
changed data values. 

15. The method of claim 14 further including transmitting a descriptor 
packet at a predetermined interval over the network, the descriptor packet 
including channel descriptors for each channel in the synchronized frames. 

16. An apparatus resident on a server for synchronizing asynchronous 
time-based and motion data in a system in which the time-based data and motion 
data are transmitted by the server over a network to a client, the apparatus 
comprising: 

a data retriever for retrieving a time-based data stream and a motion data 
stream at the server, each of the streams comprising frames of data; 
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a data stream synchronizer for buffering one of the time-based data stream 
and the motion stream to produce two streams having synchronized frames; and 

a packetizer for packaging synchronized frames of motion data and time- 
based data for use at the client for playback of synchronized motion and time- 
based data to a user. 

17. The apparatus of claim 16 further including a multicaster for 
multicasting the synchronized motion and time-based data to clients couple to the 
network. 

18. The apparatus of claim 16 wherein the packetizer includes a storage 
device and a comparator, the storage device for storing data values last transmitted 
over the network for each channel in each of the synchronized frames, the 
comparator for comparing data values for new frames with the data values stored 
in the storage device, the packetizer only packaging for transmission to the client 
channel data for channels having changed data values as determined by the 
comparator. 

19. A method for playing back time-based and motion based data that 
has been synchronized comprising: 

mapping the motion based data to control the movement of a virtual figure 
in a scene displayed at a client; and 

playing back in synchronization with movement of the virtual figure the 
time-based data. 
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20. A method of synchronizing asynchronous motion and audio data at a 
server computer in a system in which the motion and the audio data are 
transmitted by the server computer to one or more clients, the clients providing a 
real time output of synchronized motion and audio data, the method comprising: 

retrieving an audio stream including voice data and a motion data stream 
including one or more motion data channels at the server, each stream including 
frames of data; 

calculating a delay through the server for a frame of data on each of the 
streams; 

calculating a difference between the delay for the audio stream and the 
motion data stream to determine which of the two streams is faster; 

variably buffering a faster of the streams to synchronize the audio stream 
and the motion data stream resulting in two output streams having synchronized 
data frames; 

packaging the synchronized data frames; 

multicasting the synchronized data frames to one or more clients over a 
network; and 

at each client computer, using the synchronized data frames for 
synchronous playback of the audio and motion data for display to a user. 
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